E4 — Machine Learning

Machine learning’s focus on ill-defined problems and highly flexible methods makes it ideally suited for KDD applications. Among the ideas machine learning contributes to KDD are the importance of empirical validation, the impossibility of learning without a priori assumptions, and the utility of limited-search or limited-representation methods. Machine learning provides methods for incorporating knowledge into the learning process, changing and combining representations, combatting the curse of dimensionality, and learning comprehensible models. KDD challenges for machine learning include scaling up its algorithms to large databases, using cost information in learning, automating data pre-processing, and enabling rapid development of applications. KDD opens up new directions for machine learning research, and brings new urgency to others. These directions include interfacing with the human user and the database system, learning from non-attribute-vector data, learning partial models, and learning continuously from an open-ended stream of data. E4.1 Use of machine learning methods for KDD Machine learning is characterized by a focus on complex representations, ill-defined problems, and search-based methods. Representations studied include most of those described in Section B2, but particularly decision trees, sets of propositional or first-order rules, sets of instances, clusters, concept hierarchies and probabilistic networks. Ill-defined problems studied include generalizing from a set of tuples in the absence of a known model structure [C5.1], clustering [C5.5], combining logic theories of a domain with learning (de Raedt, 1996), and learning from delayed feedback in very large decision spaces (Sutton & Barto, 1998). Search methods [B8] used for learning include greedy search, gradient descent, expectation maximization, genetic algorithms, and some forms of lookahead and pruned breadth-first search. Other types of search frequently found in artificial intelligence, like best-first search and simulated annealing, tend to see less use in machine learning, for reasons discussed below. The flexibility of most machine learning methods makes them well suited to applications where little is known a priori about the domain, and/or relevant knowledge is hard to elicit. This flexibility also means they are often able to successfully learn from data that was not gathered by a purposely designed experimental procedure, but rather obtained by some process whose end goal was not necessarily knowledge discovery. The flip side of this is that theoretical analysis of machine learning methods is often difficult, and strong guarantees regarding the correctness of results are